Add HTML prerendering for FileDef entries during indexing #3918
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extend file indexing with a render phase that creates a FileDef instance and captures its display format templates (isolated, embedded, fitted, atom, head, icon) as prerendered HTML, stored in the existing boxel_index HTML columns. Rendering is non-fatal — if it fails, file metadata is still stored without HTML. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Preview deployments |
- Remove unused fileDefCodeRef destructuring in render route - Fix prettier formatting in card-prerender, prerender-app, render-runner - Add 'file-render' to kind union type in prerender-proxy-test - Fix shouldRetryWithClearCache signature line wrapping Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The extract phase already consumes the clear-cache flag, so the render phase doesn't need to consume it again. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Files with executable extensions (modules) and card JSON files already have their own prerender paths. Running the FileDef HTML render phase for these files added ~168 extra Puppeteer page transitions during boot indexing, causing CI test timeouts. Now only non-code, non-card files (like .txt, .png, .md) go through the file render phase. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ndexing" This reverts commit b435211.
The new file HTML render phase adds Puppeteer page transitions for every file during boot indexing. On CI hardware this pushes the total indexing time past the previous 60s limit. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Each Puppeteer page transition costs 2-3 seconds, and rendering all 6+ formats (isolated, head, atom, icon, fitted, embedded) for every file during boot indexing made total time O(files × formats × 3s). With ~28 files in test realms, this exceeded test timeouts. Reduce file render to isolated HTML only. Additional formats can be added back once the rendering pipeline is optimized for batch/parallel operation. The FileRenderResponse type and index columns remain unchanged — they simply receive null for the deferred formats. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… file renders Three root causes of realm-server test timeouts: 1. Missing /prerender-file-render proxy route in manager-app.ts caused boot servers to 404 when calling prerenderFileRender through the manager. 2. expectedId mismatch in render-runner.ts: captureResult expected URL without .json extension but the render route sets cardId to the raw URL including .json, causing each .json file to timeout at 30s waiting for a DOM match. 3. Module files (.gts/.ts/.js) got redundant file renders via indexFile() since visitFile() always falls through to indexFile() after indexModule(). Added hasModulePrerender flag to skip file render for modules that already produce HTML through their module prerender path. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The three root causes of the timeout have been fixed (missing manager proxy route, expectedId mismatch, redundant module file renders), so the inflated 180s timeout is no longer needed. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
Author
|
@codex Review please |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c6ddf539e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+264
to
+268
| let instance = (await cardApi.createFromSerialized( | ||
| resource, | ||
| doc, | ||
| resource.id ? new URL(resource.id) : undefined, | ||
| )) as unknown as CardDef; |
There was a problem hiding this comment.
The fileRender path instantiates the card via cardApi.createFromSerialized without passing a store, which means it falls back to FallbackCardStore (see packages/base/card-api.gts) that fetches related resources with the global fetch and no auth context. If a FileDef template touches linked/computed fields that need realm auth, those loads will 401 or be skipped, so prerendered HTML can be incomplete or incorrect. The normal render path wires instances through this.store.add/store.loaded() to ensure authenticated loading; this new branch should do the same or pass the render store into createFromSerialized.
Useful? React with 👍 / 👎.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
boxel_indexHTML columns (no DB migration needed)fileRenderflag toRenderRouteOptionsandFileRenderResponse/FileRenderArgstypes to thePrerendererinterfaceprerenderFileRenderAttemptto the render runner, with retry-with-clearCache support/prerender-file-renderto the prerender servercard-prerender.gtsfileRenderPrerenderTaskCloses CS-10124
🤖 Generated with Claude Code